Deliverable B5: A Report on Synergies with Model-based Technology and other AI Approaches in Biomedicine

descriptions of state make it possible to have more concise representations of behaviour. However, the generation of the behaviour from such descriptions tends not to produce a unique solution. This, of course, is to be expected, as the information required to produce a unique description has been eliminated in the intentional abstraction. Therefore, with respect to the real-valued description, qualitative models may produce ambiguous descriptions of the behaviour. However, such ambiguous behaviour contains useful information. For example, if it is required to predict whether the current state can lead to a critical or dangerous condition, it may be sufficient to show that none of the possible behaviours leads to a critical situation. It is important to show, therefore, that the set of possible behaviours includes the ‘actual’ behaviour of the system i.e. that the inferences made are sound. In this way, a purpose can be satisfied even with incomplete descriptions. Whereas, in traditional methods, all of the information needs to be available, and it needs to be precisely and uniquely characterised, before any inference can be made. Dynamic system simulation is an increasingly important form of qualitative reasoning. The pioneer of this approach has been the QSIM algorithm of Ben Kuipers [29]. This approach epitomises the constraint ontology, and utilises sets of qualitative equations with a landmark quantity space. One of the attractive features of QSIM is that it is designed to handle incompleteness in the knowledge of the model. The incompleteness here takes the form of a lack of knowledge concerning functional relations in the system. This situation is captured by the monotonic function constraints and between two variables, which declares that one variable monotonically increases (+) or decreases (-) with respect to another variable, covering families of relations. The simulation algorithm is a particular form of the above generate-and-test procedure from the current state assuming only that the variables are continuous. These successor states are then checked for consistency with the modelling constraints and conflicting states are eliminated. Thereby directionality of the constraints is not required and hence no causal interpretation of the algebraic constraints of the model can be derived. The algorithm can, however, be shown to predict the ‘correct’ behaviour, interpreted as the qualitative abstraction of the real-valued solution of the associated differential equation, thereby associating the causality with the integration part of the process as captured in the transition rules of the simulation engine. On the other hand, one of the well recognised aspects of human reasoning is that it is inherently vague. It appears that the use of such terms as “quite high” and “usually” are MONET Biomedical Synergies document Version 3.0 13 MONET2 Project – MONET B5 v.3.0 (Release) essential to coping with the uncertainties and complexities found in real-world situations. Any formalism purporting to capture such knowledge must, therefore, support the representation of and reasoning with such vague and imprecise knowledge. One way to represent vagueness is by the use of Fuzzy Sets [50]. These were proposed in the middle of the 1960’s by Lotfi Zadeh, and after an early flourish and a quiet 1980’s have become very popular in the 1990’s, due to the adoption of the techniques by the Japanese within domestic products. A complete theory of fuzzy logic has been developed. This allows the negation, conjunction and disjunction of fuzzy sets through the use of max and min operators that can be seen as a generalisation of classical crisp logical connectives. Fuzzy reasoning can also be supported by the compositional rule of inference that allows sets in different universes of discourse to be related through a set of fuzzy relations. This forms the basis of a mechanism for fuzzy rule-based systems where antecedents are represented by discrete fuzzy sets and the strengths of relationships, determined from fuzzy rules, are used to produce fuzzy consequents. This mechanism is, in fact, very easy to implement, using a combination of maximum and minimum of vectors of real-numbers. Indeed VLSI devices are now available that are designed specifically for fuzzy reasoning and many software systems are now commercially available that support the development of fuzzy rule-based systems. Many biomedical applications of fuzzy logic feed-back control systems are known, in fact, this was the first real application of fuzzy sets [50]. Now they are used for pattern matching, interface design and for the simulation of systems. Figure 3: A single compartment system The distinct advantage of fuzzy rule-based systems is that they are an explicit attempt to represent human expertise and are easily implemented. They may be considered as black box models; that is, once the fuzzy values have been adequately tuned the input / output relations can provide a good description of the behaviour of the represented system. The disadvantage is that they cannot provide a justification for there results because they lack structure; if they work they appear to work well, but it is difficult to establish the limits of their operability. The properties of fuzzy rule-based systems differ from those of the rule-based systems only in the representation of precision and uncertainty. 4.2 Synergies between Qualitative Models and Fuzzy Systems 4.2.1 Fuzzy Qualitative Simulation In order to achieve the explanatory power of structured qualitative models combined with the advantage of increased precision in the constitutive relations of the qualitative model, Fuzzy Qualitative Systems were developed [46, 12, 13]. Fuzzy qualitative models are structurally the same as qualitative models. However, they differ in that the variables of the model take their values from a fuzzy quantity space and the and relations are replaced by fuzzy rule bases. That is: a fuzzy rule based system forms a part of the structure of a qualitative model. MONET Biomedical Synergies document Version 3.0 14 MONET2 Project – MONET B5 v.3.0 (Release) One example of a fuzzy qualitative simulator is the Morven toolset. This is a constraintbased fuzzy qualitative reasoning system which built on the seminal work of Shen & Leitch; it contains a number of simulation and envisionment algorithms. The development of this toolset has permitted the suitability of different techniques to be examined in a number of contexts; and the comparison of different approaches to constraint-based fuzzy qualitative simulation to be made [12]. As an example consider the case of Fuzzy Vector Envisionment. This may be illustrated by means of the simple example of a single compartment system as shown in figure 3. This system is first order and can be represented by a first order differential equation of the form: from which a qualitative differential equation may be abstracted. Here is the concentration of substance in the compartment, is the infusion into the compartment and k10 the rate of elimination from the compartment If one were to model this system purely qualitatively, the situation where the rate of infusion were constant could only be given by a value of ’positive’ or ’+’ for . This situation yields very useful results in the form of a global picture of the possible behaviours of the system at the lowest precision. The integration of fuzzy sets with qualitative simulation enables the generation of envisionments at higher levels of precision. The results of one such are shown in figure 4 where the complete envisionment for a steady infusion to the compartment is the union of the envisionment graphs for each possible value of the magnitude of : p − small, p − medium, p − large or p − max in this case. Figure 4: A fuzzy envisionment of a single compartment system MONET Biomedical Synergies document Version 3.0 15 MONET2 Project – MONET B5 v.3.0 (Release) This approach to the merging of fuzzy and qualitative techniques captures strengths from both, as shown above, and permits the incorporation of temporal and empirical reasoning within a qualitative simulation in an integrated manner. 3.2.2 Learning System Dynamics through Fuzzy Models In the fuzzy modelling approach, the identification of the nonlinear system dynamics deals with the reconstruction of functional relationships between the input-output variables from the available data samples only. The function is unknown, and the modelling problem is tackled by looking for a continuous function approximator of In this modelling framework, the functional form of results from the mathematical interpretation of the fuzzy operators of M fuzzy rules: such rules are expressed through ifthen rules, and describe the knowledge on the relations between the input-output variables. The components of model parameter vector , which are tuned on the experimental data through optimization procedures, are associated with the membership functions of inputoutput variables or, in other words, with the locations of their fuzzy partition. Therefore, the modelling problem deals with the determination of the fuzzy partition of the input-output variables, and the optimal number of rules that must be used to generate the fuzzy model. In theory, both partitions and inference rules can be derived by the expert knowledge, but such information may be very poor, irregular and unstructured, and then, in practice, prevents one from defining the optimal form of , where by optimal we mean that is of minimal complexity, but able to capture all of the significant features of the system dynamics. For these reasons, the research efforts turned to the definition of learning methods that automatically generate the fuzzy systems from the data samples only [47]. Although these methods have been successfully applied to a variety of domains, they are affected by two serious drawbacks. Let us observe that the same problems may occur even when the resulting model is abstracted from the expert knowledge, since an empirical rather than structural kind of knowledge is mostly given. A part serious inefficiency problems due to the possible combinatorial explosion of the number of rules and parameters when the number of inputs increases, these methods suffer from lack of robustness and interpretability. As for robustness, the following problems may occur: 1. generalization capability of the built model may be lost as overfitting phenomena may occur, especially when the data set is not large enough as it may often occur in the biomedical domain; 2. numerical instability phenomena during parameter estimation, e.g. solutions do not necessarily depend on the data in a continuous way, may occur. In addition, fuzzy models generated from data only do not guarantee to gain insight into the system: the resulting model structure is often not transparent, and, after their optimization, the model parameters, that are associated with the locations of the fuzzy sets rather than with the biomedical reality, may lead to an incomplete, inconsistent and even indistinguishable fuzzy partition. MONET Biomedical Synergies document Version 3.0 16 MONET2 Project – MONET B5 v.3.0 (Release) Integration of Qualitative Models and Fuzzy Systems: a hybrid method Let us observe that for a great deal of pathophysiological systems, the available structural knowledge is insufficient for the formulation of a quantitative differential model, but does not prevent the formulation of a qualitative one. This consideration motivated the definition of a new hybrid approach, called FS-QM, to the fuzzy identification of dynamical systems [5,6]. Its novelty consists in the way the fuzzy model is built: both the fuzzy partition and rule base are defined upon the available structural knowledge. FS-QM is applicable whenever the incompleteness of a priori knowledge is such that it allows us (i) to write a QSIM model [29], and (ii) to bound the uncertainty on landmark values to a range of numerical values. In outline, the whole range of possible system dynamics, represented and simulated within the qsim modelling framework, is automatically translated into the fuzzy formalism. The domain of each input/output variable is automatically partitioned into fuzzy sets in accordance with its associated quantity space, and with the prior information on landmark numerical bounds. In other words, the cardinality of the fuzzy partition of a variable and the membership function locations are defined by the cardinality of the set of qualitative values the variable may assume, and by the interval bounds of its landmarks, respectively. Given a landmarkbased fuzzy partition and the simulated behaviour tree, the generation of the fuzzy rule base is straightforwardly derived by mapping each behaviour of the input/output variables into a set of rules, where each rule describes a transition from a qualitative state to the next one. The mathematical interpretation of the rule base explicitly initializes which is then refined through parameter identification from the data samples. Fig. 5 sketches the main steps of FS-QM. Figure 5: Main steps in FS-QM Efficiency, robustness, and interpretability of FS-QM models The hybrid method outlined above, where the structure identification phase is clearly separated from the parameter optimization one, allows us to get a nonlinear functional relationship between input and output variables that is both robust and interpretable [20]. It allows us to get (i) good generalization properties as the fuzzy system does represent the system dynamics, (ii) numerical stability as, due to the a priori knowledge and the particular way we perform variable partitions, regularization techniques can be applied, and, MONET Biomedical Synergies document Version 3.0 17 MONET2 Project – MONET B5 v.3.0 (Release) consequently, the search of optimal parameters is restricted within a ”trust” region. The regularized problem also allows us to get a fuzzy partition which is complete, consistent, and distinguishable after the parameters have been tuned on the data. This together with the fact that the parameters do refer to the reality makes the identified model completely interpretable. Moreover, the method is efficient as the number of both rules and parameters is linear with the cardinality of the fuzzy partition or, in other words, by the cardinality of the quantity space. Application perspectives in the biomedical domain To achieve a robust and interpretable fuzzy model, FS-QM effectively employs all the available structural prior knowledge, represented in QSIM, and empirical data. The embedding of deep prior knowledge into the function makes the identification problem better posed, since it properly delimits the model search space. In addition, the prior knowledge allows us to define a good initial estimate of the parameter vector , and then, to define a trust region where is supposed to belong. If the prior knowledge is correct, this will lead to a model that has good generalization and interpretability properties even in datapoor contexts. Although FS-QM rests on correct prior knowledge expressed as a QSIM model, and in particular on physically significant predefined landmarks, its range of applicability in the domains of Life Sciences in which we are interested is quite wide, as such knowledge is available for a great deal of dynamical systems. In particular, it has already been successfully applied to model: 1. a predictor of blood glucose level in diabetic patients [4] 2. a simulator of the intracellular thiamine (vitamin B1) kinetics in the intestine tissue [7]. Let us observe that as for the metabolic system (2), FS-QM has allowed us to solve serious identification problems: (i) the classical differential approach turned out to be inapplicable for the incompleteness of the available knowledge and for the difficulty of gathering an adequate number of experimental data;(ii) the conventional fuzzy approaches failed for the too small number of available input-output samples [7]. In FS-QM, the gained parameter interpretability from the physical point of view represents an added value that makes it possible for fuzzy models to be used to perform a larger spectrum of tasks than the usual one, namely system control. FS-QM models might be conveniently applied, for example, in a diagnostic context. On the one hand, diagnostic hypotheses that explain the observed behaviours could be tested by introducing structural variations into the underlying qualitative model, and by validating the fuzzy model built on the basis of the newly generated rule base. On the other hand, diagnostic hypotheses could be drawn from the analysis of the deviations of the estimated values of parameters from the nominal ones. 5 Qualitative Reasoning and Inductive Logic Programming The development of intelligent tools to aid in the process of Scientific Discovery, particularly in the construction of explanatory models, is an important goal of AI; and qualitative modelling provides an ideal representation. This is the ultimate in adaption, and a hybrid system merging Inductive Logic Programming and Qualitative Simulation is a suitable tool for achieving it. Bioinformatics is an ideal domain for applying this technology: the data are sparse (making it unsuitable for numerical techniques), they are noisy and they require the construction of models which will inevitably include unobserved variables. Work on constructing models of systems in molecular biology is in the early stages of development MONET Biomedical Synergies document Version 3.0 18 MONET2 Project – MONET B5 v.3.0 (Release) and so, given the above stated challenges, any useful results emerging will be of tremendous practical value. The ultimate goal in this scientific quest is the production of quantitative models; however, the discovery of suitable structural models (qualitative differential equations) can be the means of directing the scientist as to which experiments to carry out next in the path towards this goal. In this section we present a learning system which combines Inductive Logic Programming (ILP) with QSIM in order to construct qualitative models of physical and biological systems containing unmeasured variables [14]. The general model learning problem can be represented deductively as follows: if we term the observations (evidence) E, the background knowledge B, and the hypothesis to be learnt H, then given that: